National action plans for antimicrobial resistance and variations in surveillance data platforms

Abstract Objective To assess how national antimicrobial susceptibility data used to inform national action plans vary across surveillance platforms. Methods We identified available open-access, supranational, interactive surveillance platforms and cross-checked their data in accordance with the World Health Organization’s (WHO’s) Data Quality Assurance: module 1. We compared platform usability and completeness of time-matched data on the antimicrobial susceptibilities of four blood isolate species: Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus and Streptococcus pneumoniae from WHO’s Global Antimicrobial Resistance and Use Surveillance System, European Centre for Disease Control’s (ECDC’s) network and Pfizer’s Antimicrobial Testing Leadership and Surveillance database. Using Bland–Altman analysis, paired t-tests, and Wilcoxon signed-rank tests, we assessed susceptibility data and number of isolate concordances between platforms. Findings Of 71 countries actively submitting data to WHO, 28 also submit to Pfizer’s database; 19 to ECDC; and 16 to all three platforms. Limits of agreement between WHO’s and Pfizer’s platforms for organism–country susceptibility data ranged from −26% to 35%. While mean susceptibilities of WHO’s and ECDC‘s platforms did not differ (bias: 0%, 95% confidence interval: −2 to 2), concordance between organism–country susceptibility was low (limits of agreement −18% to 18%). Significant differences exist in isolate numbers reported between WHO–Pfizer (mean of difference: 674, P-value: < 0.001, and WHO–ECDC (mean of difference: 192, P-value: 0.04) platforms. Conclusion The considerable heterogeneity of nationally submitted data to commonly used antimicrobial resistance surveillance platforms compromises their validity, thus undermining local and global antimicrobial resistance strategies. Hence, we need to understand and address surveillance platform variability and its underlying mechanisms.


Introduction
Antimicrobial resistance is a growing threat to global public health. 1 Recognizing the need for coordinated, evidence-based action, the 2015 World Health Assembly endorsed the Global action plan on antimicrobial resistance, 2 with Member States agreeing to mandate the development and implementation of national action plans on antimicrobial resistance aligning human, animal and agricultural measures.
Timely, accurate, relevant data are fundamental to informing country measures addressing antimicrobial resistance, hence the second of the five key global action plan implementation objectives is to "strengthen the knowledge and evidence base through surveillance and research." 2 Acknowledging that different countries may be at various starting points, the World Health Organization (WHO) has subsequently helped countries establish antimicrobial resistance surveillance and encouraged them to join their Global Antimicrobial Resistance and Use Surveillance System (known as GLASS). 3 WHO also offers technical support, guidance, laboratory reporting standards and coordinating mechanisms for antimicrobial stewardship to countries needing strengthening of their diagnostic laboratory capacity. An aim of the support is to enable countries to submit clinically linked, nationally gathered data to WHO's surveillance system, to describe both current and emerging resistance, and to monitor antimicrobial resistance and national action plans interventions. 4 Initial assessment of developments of national surveillance capability following the release of the global action plan suggested some improvements, including in access to funding, but highlighted ongoing challenges and limited reporting outputs, [5][6][7] particularly in low-and middle-income countries. 8 In 2020, researchers were able to identify 71 separate international antimicrobial resistance surveillance platforms, ranging from targeted single disease surveillance, such as for tuberculosis, to supranational regional activity mirroring the aims of WHO's surveillance system. However, very few offered readily available open-access data. 9 These platforms included commercial platforms such as Pfizer's antimicrobial testing leadership and surveillance database, which provides user-friendly, open-access and interactive visualization of available data, and has recently announced a public-private collaboration with the Wellcome Trust to address antimicrobial resistance in sub-Saharan Africa. 10 As the coronavirus disease 2019 (COVID-19) pandemic comes under control, antimicrobial resistance must return to the forefront of the global health agenda. The pandemic has led to deterioration of antimicrobial susceptibility reporting activities 11,12 and many of the national action plans have now expired. Now is an important moment to identify the current issues in global progress so that we can optimize the effectiveness of future actions; thus we need to evaluate the current surveillance platforms. We therefore analysed and compared international open-access antimicrobial resistance surveillance systems, using the WHO data quality assurance framework, dimension 3, that is, external comparison and/or cross-checks with other data sources. 13 This analysis included assessing the consistency of the platforms' data output of key pathogens.

Methods
We conducted a search to identify potential, supranational, open-access, antimicrobial resistance interactive platforms for comparison with WHO's global antimicrobial resistance and use surveillance system 2019 data (latest available year of reporting at the time of the search). The search was initially conducted in October 2021 and repeated in July 2022. First, we screened the 71 international antimicrobial resistance surveillance platforms identified in a 2020 review 9 for suitability. We then searched the individual Member States' health ministry (or equivalent) websites for involvement in additional supranational schemes. We screened the individual national action plans that were available in the WHO library of antimicrobial resistance national action plans 14 for mentions of additional specific platforms. Finally, we conducted a general internet search using the Google search engine and the search words "AMR", "antimicrobial resistance", "national action plan", "NAP" and the specific country of interest.
We used the following inclusion criteria: the platform had to (i) be entirely open access, interactive and web-based for reporting and visualizing antimicrobial resistance data; (ii) have data available to compare to those of 2019; (iii) represent at least supranational reporting of regional data; and (iv) contain data on blood culture isolates. The exclusion criteria were: not having openaccess data via a readily open-access interactive platform; having no data available on the study period; or only partial reporting of data (organism of interest but not suitable antimicrobial).

Analysis of surveillance data
For comparisons, the WHO data quality assurance framework suggests selecting a core set of four to five tracer indicators to identify any data completeness and quality issues. 13 Thus, to enable direct comparison with other databases, we searched the WHO global antimicrobial resistance and use surveillance system for resistance data on four key blood stream infection organisms represented across the platforms: Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus and Streptococcus pneumoniae. The 2021 Global antimicrobial resistance and use surveillance system (GLASS) report states that the data collected for each data call (the last was in 2020 for participating countries) are antimicrobial susceptibility rates for the previous calendar year. 15 We extracted the data on the number of isolates submitted for each species, the antimicrobial susceptibility results, age and gender of patients, number of patients tested and the origin of infection for each isolate. We then categorized these according to the system's parameters of (i) no data available; (ii) < 70% data reported; or (iii) 70%-100% data reported. We also extracted the reported antimicrobial susceptibilities for the available indicators of resistance. For E. coli and K. pneumoniae, we selected the third-generation cephalosporin ceftazidime (or when not available, ceftriaxone); for S. aureus, oxacillin (or when not available, cefoxitin); and for S. pneumoniae, penicillin (or when not available, oxacillin). We selected the alternative antimicrobial when the primary selection was not being reported, or less than 30% of isolates having sensitivity results were available for primary selection. Six of the authors extracted these data across each identified platform, a different author covered each WHO region, and one author cross-checked all the regions.

Comparison of platforms
To compare the strengths and weaknesses of platforms identified, we used pre-defined criteria. These criteria consisted of a broad overview of a combination of WHO Data Quality Assurance framework dimensions (qualitative consideration of data completeness, timeliness and internal consistency) 13 and features specific to platform use, such as data accessibility and extraction, data representation and platform usability. We also pooled and summarized the qualitative comments from the data extractors to identify any strengths and weaknesses in visualization of data between platforms. Finally, we created a minimum recommended data set template as a potential method for increasing antimicrobial resistance reporting, engagement and representation.

Statistical analysis
We conducted the statistical analysis and data visualizations in R version 4. 1.1 (R Foundation, Vienna, Austria), using the tidyverse, gtsummary, sf and rnaturalearth packages. We summarized the categorical variables as frequencies and percentages, and the continuous variables as medians and interquartile ranges (IQRs). We also stratified the countries' key variables by WHO region.
We used Bland-Altman analysis to assess concordances between the proportion of isolate susceptibility that each country reported to WHO's and identified platforms. We matched each organism with each country (hereafter referred to as organism-country combinations). This technique quantifies the concordances between two continuous measurements by calculating the mean difference (bias) and constructing limits of agreement (within which lie 95% of the differences between measurements). 16 We then used paired t-tests to assess whether each country reported different mean susceptibility percentages for each organism to the two platforms. The number of isolates that each country reported to different platforms was summarized using medians and the median of the differences. We then compared these using Wilcoxon signed-rank tests to account for the paired data.

Identification of platforms
We did not identify any additional platforms other than the 71 previous identified platforms.
In addition to WHO's surveillance system, Pfizer's antimicrobial testing leadership and surveillance database met the inclusion criteria and had a global scope. The European Centre for Disease Prevention and Control's Variations in antimicrobial resistance surveillance platforms Scott JC Pallett et al.
(ECDC's) European antimicrobial resistance surveillance network was the only regional platform that met the inclusion criteria. Both WHO's and Pfizer's platforms enable the analysis of blood stream infection isolates independently of other specimen types, making possible direct comparison of the reported susceptibility rates for 2019 across countries. The ECDC network combines data on blood stream infections and cerebrospinal fluid. As the ECDC network feeds directly into WHO's system, the aim of the comparison was to assess whether combining reported susceptibility estimates of important blood stream isolates and cerebrospinal fluid together resulted in any significant variance in reported organism susceptibility between the two platforms.

Surveillance platform activity
As of August 2022, a total of 103 of the 194 (53.1%) WHO Member States have enrolled in WHO's surveillance system. Of these, 100 (97.1%) have signed up to submit antimicrobial resistance surveillance data, and 18 (17.5%) have signed up to submit antimicrobial consumption data (Fig. 1). Of the 100 countries that committed to submit antimicrobial resistance surveillance data, 67 (67.0%) do so, with a further one country submitting partial data (1.0%). Three countries that have not enrolled also submit data (70/194; 36.1%; Fig. 1). Of the 71 countries actively submitting data to WHO's surveillance system, 28 (39.4%) also submit to Pfizer's platform and 19 (26.8%) submit to ECDC. Sixteen countries (22.5%) submit to all three platforms.

Surveillance data quality
Countries reporting on the four pre-set organisms and their associated antimicrobial sensitivity are presented in Table 1 (available at https:// www .who .int/ publications/ journals/ bulletin/ ). Examining the proportion of organism-country combinations that had 70%-100% data reported to WHO's surveillance system, we found that: 96.8% (271) of combinations had antimicrobial sensitivity data; 88.9% (249) had information on gender; 83.6% (234) had information on age; 35.7% (100) had information on the total numbers of patients tested; and only 21.4% (60) had information on infection origin. The Western Pacific and African Regions provided data more consistently on the numbers of patients tested; the South-East Asia, European, Western Pacific Regions provided data on age, and the Western Pacific Region provided data on infection origin. Across the Regions of the Americas, the reliability of the available sensitivity and age data was comparatively low, whereas in the European Region, the reliability of the available infection origin data was notably low (available in the online repository). 17 Across WHO regions, significant variation was noted in the susceptibility data regarding E. coli, K. pneumoniae and S. aureus, but less variation regarding the S. pneumoniae data ( Table 2).
Comparison of the platform data showed that the data submitted to WHO's surveillance system were more antimicrobial susceptible than average data submitted to Pfizer's platform (bias: 4%, 95% confidence interval, CI: 1 to 7). The concordance between these two platforms' organism-country susceptibilities was extremely low, with 95% limits of agreement ranging from −26% to 35%. This result indicates that for 95% of organismcountry combinations, the absolute difference between the susceptibility reported to WHO's surveillance system and that reported to Pfizer's platform was possibly as great as 35% (Fig. 2). We found no evidence that WHO's and ECDC's surveillance platforms had different mean susceptibilities (bias: 0%; 95% CI: −2% to 2%). However, the concordance between the organism-country combinations was low, with 95% limits of agreement from −18% to 18%, even though two outlying data points primarily drove this result (Table 3).
We found significant evidence that countries report different numbers of isolates to WHO's surveillance system and Pfizer's platform (P-value: < 0.001), and significant evidence that countries report different numbers of isolates to the WHO and ECDC platforms (P-value: 0.04). Comparison of the number of isolates reported to the WHO and Pfizer platforms revealed that the median of the differences was 674 isolates (IQR: 175 to 1917 isolates). Comparison of the number of isolates reported to the WHO and ECDC platforms revealed that the median of differences was 192 isolates (IQR: −273 to 1743 isolates). Table 3 presents a summary of statistics stratified by organism. Table 4 presents the overall aims of each platform, and their weaknesses and strengths regarding consistency in presentation and accessibility of data; reporting standards; completeness and quality of data; and consistency of data across key demographic indicators.

Proposed data set requirements
As we found that the data representativeness and data quality vary across the platforms and WHO regions, we propose a minimum data set requirement for reporting blood stream infection antimicrobial resistance data in the form of a potential template (Table 5). This template focuses on reporting at least the four blood stream infection organisms analysed here alongside the key antimicrobial susceptibility indicator data and the baseline demographic data.

Discussion
Our findings suggest considerable inconsistencies between the surveillance data in supranational observatory platforms, raising concerns about their reliability for reflecting national or local community needs. In 2021, WHO announced a renewed Call to action on antimicrobial resistance, seeking to accelerate the commitments made previously to tackling this global public health concern, using the One Health approach but considering the varied circumstances of individual countries. 19 Having garnered the active support of 113 Member States, an opportunity now exists to identify and address the deficiencies in antimicrobial resistance surveillance data.
Making flexible, open-access antimicrobial resistance surveillance platforms that require minimum entry available to reporting laboratories to facilitate accuracy, rather than striving for unachievable completeness in surveillance data submission, could enable countries lacking the diagnostic or workforce capacity to obtain meaningful surveillance data for national measures and international collaboration. 20 The substantial discrepancies between surveillance platforms in species susceptibility within countries revealed here reduces the ability to reliably monitor any development in  We obtained evidence that 71 countries submitted surveillance data during the global antimicrobial resistance and use surveillance system's 2020 data call. Countries that are enrolled in the system but have no data for the 2020 data call are also highlighted on the map.
Variations in antimicrobial resistance surveillance platforms Scott JC Pallett et al. national, regional and global antimicrobial resistance patterns. This variability must be addressed without delay if we are to ensure reliability of private or public platform outputs and to avoid misdirecting antimicrobial stewardship and research on antimicrobial resistance and antimicrobial stewardship at the national and regional levels. 10,21 The wide variation between countries in the amount of species data submitted to each platform highlights sample selection bias. In addition, smaller sample sizes are unlikely to represent any variability in inter-city or regional resistance. [22][23][24] To improve the submission of reliable data, we suggest that laboratories should be provided with a minimum required reporting data set template that includes only key pathogens. This approach may be especially useful in invigorating surveillance activity in those countries whose capabilities are still in the early development stage. This template could also stipulate that only the susceptibility of indicator antimicrobials is required (as in the ECDC's network), which would help countries focus on susceptibility testing strategies when funding is scarce but allow for regional variation in the selection of appropriate/available indicator antimicrobial agents. WHO has recently published methodological principles for nationally representative surveys of antimicrobial resistant blood stream infections, 25 which may be further facilitated by a minimal data set approach. While improving diagnostic capability is likely to require substantial financial investment in some situations, this document provides timely guidance for countries with limited surveillance infrastructures to undertake periodic strategic sampling of defined population subsets to address reporting bias issues. 25 This approach could be combined with restricting  Mean susceptibility between platforms, % Bias 95% CI Upper limit of agreement Lower limit of agreement CI: confidence interval; ECDC: European Centre for Disease Prevention and Control; WHO: World Health Organization. Note: Included databases are WHO's global antimicrobial resistance and use surveillance system, Pfizer's antimicrobial testing leadership and surveillance and ECDC's European antimicrobial resistance surveillance network. The y-axes show the differences between the susceptibilities of each organism-country combination result (i.e. the difference between the E. coli susceptibility to third-generation cephalosporins for Japan reported to WHO's system, and those reported to Pfizer's platform).
Variations in antimicrobial resistance surveillance platforms Scott JC Pallett et al.
national data reporting requirements to a minimum and optimizing available funds to ensure adequate diagnostics to support this minimum data set. Subsequently, platforms should be adapted to include information on source data type (periodic survey versus routine national data) and should streamline upload mechanisms for minimum versus expanded data sets. Sharing the lessons learned with regional partners and considering the adoption of a periodic survey method potentially coordinated by the regional WHO offices will be integral for maximizing efforts and avoiding duplication of work. Although capacity strengthening is essential for developing surveillance platforms, giving a clinical context to the available data could also be a priority for established platforms. 5 A major benefit of WHO's surveillance system is the option to submit isolate-level clinical information, and although demographic data are often available, information on infection origin (particularly in Europe) and the total number of isolates tested is often lacking. Combining clinical information and antimicrobial resistance data can improve the scope and applicability of individualized antimicrobial stewardship guidelines. 20 Even accounting for the additional time and resource burden associated with submitting data to WHO's surveillance system in a tertiary hospital in Thailand, for example, the authors consider WHO's system outputs superior in contributing to antimicrobial guideline development. 20 Accurate interpretation of the variation in bacteraemia isolation rates during COVID-19 has been complicated by imprecise denominator estimates, even in countries that are able to provide the most comprehensive data, and this highlights the importance of improving data quality across the board. 26 Multiple platform use is likely to further challenge the already limited workforce capacity, and if opportunities to optimize data quality are not taken, alternative platforms could seek to support the visualization of WHO's system data through enabling submission via a single platform or through providing a specific function, rather than relying on comparatively limited data to address present inconsistencies. At the very least, platforms should provide an opportunity to compare data by individual specimen type, as evidenced by the observed variation in the isolate data in the WHO's and ECDC's platforms, despite reporting via a sophisticated platform using national data.
Although we were able to evaluate comparators, open-access platforms against all the available WHO's system data, we acknowledge that some countries also engage in further closed surveillance networks (such as the Asian network for surveillance of resistant pathogens), semi-open access networks that look at a limited number of organisms (such as gram negative surveillance by the global study for monitoring antimicrobial resistant trends) or belong to networks that provide regular reports but have no interactive platform (Central Asian and European surveillance of antimicrobial resistance network).
Our results raise concerns about the heterogeneity of the matched country data of some of the most established observatories. We recommend that those seeking to inform policy consider further evaluating the data held within these restricted-access networks. Our findings also reveal data discrepancies during the last full year of reporting before the COVID-19 pandemic, followed by a period of increased antimicrobial use and diverted laboratory capacity. These backdrops are highlighting a need to urgently improve data reliability across platforms to understand the true impact of the COVID-19 pandemic on global antimicrobial resistance. When evaluating the surveillance strategy in their specific regions, policy-makers should bear in mind that in some areas,  current reporting capacity is likely to be more limited.
In conclusion, the surveillance data submitted to various supranational antimicrobial resistance monitoring platforms seem to be significantly heterogeneous, which may compromise their validity and undermine national and global strategies. This heterogeneity is particularly concerning for low-and middle-income countries as misinforming of their decision-makers may affect the perceived need for specific diagnostics or antimicrobial guidelines.
Policy-makers must be made aware of the potential unreliability of the platforms intended for informing strategy or outcomes. Mitigation measures must be taken to reduce surveillance bias through limited reporting and improve the ability to report more representative data in the short-term. These measures are particularly relevant in countries that need to improve their national surveillance platforms. Recent WHO recommendations to consider periodic strategic surveys in such circumstances seek to address this issue and may be further complimented if a minimum required data set is agreed on to streamline reporting and optimize representation in the short-term. ■

Planes de acción nacional orientados a la resistencia antimicrobiana, y variaciones en las plataformas de datos sobre vigilancia
Objetivo Analizar cómo difieren los datos nacionales sobre susceptibilidad antimicrobiana utilizados para conformar los planes nacionales de acción, entre las diferentes plataformas de vigilancia. Métodos Se i dentificaron las plataformas de vigilancia disponibles, de libre acceso, supranacionales e interactivas, y se llevó a cabo una comprobación cruzada de sus datos, de conformidad con el Control de calidad de datos: módulo 1 de la Organización Mundial de la Salud (OMS). Se realizó una comparación entre la utilidad de la plataforma y la exhaustividad de los datos, coincidentes en el tiempo y relativos a la susceptibilidad antimicrobiana de cuatro tipos de bacterias sanguíneas aisladas: Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus y Streptococcus pneumoniae, información procedente del Sistema Mundial de Vigilancia de la Resistencia a los Antimicrobianos de la OMS; de la red del Centro Europeo de Control de Enfermedades (ECDC) y de la base de datos de Liderazgo y Vigilancia de Pruebas Antimicrobianas de Pfizer. Utilizando el análisis de Bland-Altman, las pruebas t pareadas y las pruebas de los rangos con signo de Wilcoxon, se analizaron los datos sobre susceptibilidad y el número de concordancias aisladas entre plataformas.